Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity
نویسندگان
چکیده
Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how local learning rules at single synapses support behaviorally relevant adaptive changes in complex networks of spiking neurons. However the potential and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allow us to predict under which conditions reward-modulated STDP will be able to achieve a desired learning effect. In particular, we can produce in this way a theoretical explanation and a computer model for a fundamental experimental finding on biofeedback in monkeys (reported in [1]).
منابع مشابه
Functional requirements for reward-modulated spike-timing-dependent plasticity.
Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term th...
متن کاملReinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity
The persistent modification of synaptic efficacy as a function of the relative timing of pre- and postsynaptic spikes is a phenomenon known as spike-timing-dependent plasticity (STDP). Here we show that the modulation of STDP by a global reward signal leads to reinforcement learning. We first derive analytically learning rules involving reward-modulated spike-timing-dependent synaptic and intri...
متن کاملReward-modulated spike-timing-dependent plasticity with a dynamic spike timing rule and inhibitory plasticity
The viability of spike-timing-dependent plasticity (STDP) to explain learning processes is controversial, although recent developments of reward-modulated STDP (RM-STDP) models provide a plausible substrate. However, evidence has also emerged to show that rewards themselves can modify the STDP rule. In this modeling study, we use a dynamic STDP rule to show that such modification can lead to ne...
متن کاملReward Modulated Spike Timing Dependent Plasticity Based Learning Mechanism in Spiking Neural Networks
Spiking Neural Networks (SNNs) are one of the recent advances in machine learning that aim to further emulate the computations performed in the human brain. The efficiency of such networks stems from the fact that information is encoded as spikes, which is a paradigm shift from the computing model of the traditional neural networks. Spike Timing Dependent Plasticity (STDP), wherein the synaptic...
متن کاملReinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons
Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modu...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2007